Software of the Month Club 1996 May

home *** CD-ROM | disk | FTP | other *** search

/ Software of the Month Club 1996 May / Software of the Month Club 1996 May.iso / mac / ISO9660 / ZIPFILES / LXLT101.ZIP / lxlite.doc < prev next >

Wrap

Text File | 1996-02-23 | 20.9 KB | 412 lines

────────────────────────────────────────────── LX lite - an compressor for OS/2 executables ────────────────────────────────────────────── Dedication: To my little daughter Alice, born 12 Feb, 1996 at 03:45. 1. Distribution ─────────────── This program is freeware. This mean that you can distribute it in any way, except for commercial use. Commercial use is allowed ONLY with my personal permission. To contact me see the the last section in this file. This also means that you have NO guaranties that it will perform exactly what YOU need. I cannot carry any responsability for any damages, lost of profit etc. caused by use or inability to use this program. However, you CAN use it to enhance any, even commercial, product. This is not for YOUR benefit but rather for benefit of poor users who are BORED of the big size of executables. The program is written exclusively in Virtual Pascal, beta 3, with use of its built-in 32-bit assembler. This is an excellent language which takes full advantages of OS/2`s possibilities, is BorlandPascal compatible, and have a powerful built-in optimizer. If you want the lxLite`s source code, please mail me, but you have to tell me WHY do you need it, to avoid the people which likes to re-sell other`s programs under their name. 2. Introduction ─────────────── I think all of us are really bored of the big size and reduced functionality of all those modern executables written to run under OS/2 (windos too). I don`t understand why they are so big, because most compilers, even IBM CSet generate a modest size code. For a widely known example let`s take MultiMaint. What a complex task it performs that its executable occupies more that 700K? I don`t understand. Moreover, WHY duplicate (and triplicate) almost the same executable as it does (I mean MultiSafe and IniMaint which comes along with MultiMaint). It performs a nice work, but it is TOO big for the task it acomplishes. OS/2 kernel have the same size. I cannot afford such a large pile of shit on my HD, so I killed MultiMaint & C°. :`-(. Too bad for its author. Here is a workaround for this. You can just pack the executable so it will be twice smaller and still perform the same task. Alas, it will grab the same amount of memory as original executable does - this is the fault of the program`s author. As far as I know there is only one program which does the same - REPACK from IBM (EWS?). But compared to lxLite it gives larger files using the same algorithm. For example, COURIER.FON from OS/2 v8.192 it compresses into 44K and lxLite into 34K. Feel the difference. BTW, LINK386+Resource Compiler compiles COURIER.FON also into 44K-size file. This make me think that they use the same common library. I packed ALL my executables (incuding but not limiting to ?:\os2\*.exe, ?:\os2\dll\*.* and ?:\os2\dll\ibmnull;laserjet) and my system is stiil working fine :-) One of lxLite users (Pavel Roskin) observed that lxLite packs even os2krnl :-) This is a very nice feature to create a SINGLE OS/2 boot diskette. 3. Features ─────────── lxLite compresses the files in the same way as LINK386 does. There is NO way of implementing a executable packing method other than those two which Warp uses (or the only for 2.x). So, here is a brief description of those two algorithmhs: 1. Run-length packing. This is generally the same method that Microsoft C for DOS uses. It gives VERY bad results because the structure of executable files aren`t suited for that kind of packing. For example, PCX files uses approximatively the same packing method. 2. Kinda Lempel-Ziv algorithm. Lempel-Ziv is the same method which almost all DOS executables packers use - LZEXE, PKLITE, PGMPAK etc. The method standartized for OS/2 executable files is not the most effective (IMHO). Other thing is that OS/2 executable have a different loading algorithm in contrast to DOS executables - parts of OS/2 executables are loaded only when they are needed. So, Lempel-Ziv dictionary cannot cross the bound of a single page (4096 bytes). Because of this, it gives not such good results as it can. lxLite can use any of these two methods either for packing or unpacking. Generally the second gives best compression rates, but may be (?) there are some files on which first will work better. Because of this the default is to try to pack page using both methods and then choose the smaller result. lxLite can be used as well for unpacking the files that previously have been packed - either by lxLite, LINK386 or REPACK from IBM. What kind of files you can compress? The LX format is used almost everywhere in OS/2: almost everything is in LX format. So, you don`t have to limit only to .EXE files: you can pack .DLL, .PDR, .QPR, .DRV, .FON, .SYS (Virtual Device Drivers (VDDs) in \OS2\MDOS) as well. You can run lxLite on virtually any file: if it is not in Linear Executable (LX) format it will refuse to process it. You can also consider compressing your entire \OS2\*\ directory structure, it will give you a lot of extra space and absolutely no overhead! The time spent for decompressing executables is recovered by the time which system does not spend for reading the executable from HD because it`s much smaller! So, reboot from stand-alone OS/2 diskettes (you can boot from OS/2 installation diskettes (first two) then press F3), then change to drive where your OS/2 is and type \[path]\lxLite os2\*.exe os2\dll\*.* os2\dll\ibmnull\*.drv and so on. Also you can compress now the executables which are run at system startup, because when they are run you cannot modify their executables and DLL`s. lxLite version 1.00 and above can replace executable modules even if they are currently in use. In this case it will warn you that module is already in use by another process, and will propose you either to replace it by its packed version or either to skip this module. Choose at your wish, but keep in mind that the modules you have replaced are kept now (their old versions) in memory, so they eat up your swapfile space. Better reboot as soon as you have this opportunity. Versions above 1.00 are supplied in two different executables: one named lxLite.exe is the usual version for OS/2 v2.99, Warp and aboves. Other, named lxLite2x.exe is meant for use in old versions of OS/2 (i.e. 2.x, NOT 1.x because 1.x doesn`t have the LX format). Delete it if you`re a OS/2 Warp user. 4. Options ────────── There are alot of options in lxLite. I simply like options :-) So, you can configure almost anything in lxLite. Moreover, to protect the user from need of writing the same options lxLite support multiple configurations which are kept in a single file. lxLite comes with some example configurations (`factory defaults`) which are listed below: ──────────────────────────────────────────────────────────────────────────── default: (loaded by default) It is a `work` configuration. All parameters are set to maximize compression. .BAK file creation is disabled. Note that this configuration is ALWAYS loaded before any other options are in effect, so even /C{#} option is executed AFTER default configuration is loaded. fast: Lowest compression level, fastest executable loading. According to IBM when data objects are aligned inside of file on sector boundary (512 bytes) executable will load faster. I cannot observe a difference, so try it at your own. If some data in executable was previously packed it is not de-packed then re-packed. fast2: Compress a bit better, but slower. Loads as fast as CFG#1 does. ver2x: Optimal for pre-Warp versions of OS/2. Versions before Warp (3.0) does not know of the Lempel-Ziv (/EXEPACK:2) method. DON`T PACK EXECUTABLES WITH LEMPEL-ZIV`s ALGORITHM if there is a chance to run your program on OS/2 v2.xx (except 2.99). failsafe: Lempel-Ziv packing enabled. A `fail-safe` configuration. All data are written just like LINK386 does, because of this creates a bit bigger files than using default configuration. FOR WARP ONLY! max: Tightest compression level. VERY SLOW! It is rarely needed to use this configuration. newStub: This is a particular configuration used to replace one DOS stub in LX executable by another without altering anything else. You have to specify a filename for new stub - this configuration only tells lxLite not to depack old objects and not to pack unpacked objects. minStub: This is a configuration which is linked to newStub and replaces stubs in given files by minimal possible stub of `say-error-and-exit` type. You cannot make it smaller (at least I doubt it), only if you shorten the error message. vdmStub: This configuration tells lxLite to replace in specified files stub by a `run-from-VDM`-style one. This is also as short as possible :-) Please read the /T{#} option description for further details. info: Use this configuration to get information about executables (see options /V+) without rewriting them. ──────────────────────────────────────────────────────────────────────────── To use a specific configuration use /C# where # is configuration`s name. The settings are stored into file lxLite.CFG in the same directory where lxLite.EXE resides. You shouldn`t care about paths - lxLite will always find it. For example, to use `max` configuration run lxLite /cMax. For a detailed description of .CFG file format see section right below the following. Here is a detailed explanation of what each switch does. Note that any switch which accepts '+' or '-' sign after it (to enable/disable the action which switch symbolizes) can be used without anything after it - this is accepted as '+'. So, /V+ is equivalent to /V. ■ /A{P|S|N{P|S}} Set alignment method for first and rest of objects. First object can be aligned on [P]age shift, [S]ector or [N]o boundary. Note that the last option (No boundary) is never used by LINK386, but it works well, and the LX format allows it. All objects except first MUST be aligned at least on a PageShift boundary. PageShift is a value that is specified in LX header. If you want to redefine it use /R# option. ■ /B{+|-} Enable (+) or disable (-) renaming of original file into .BAK. Note that this is a BETA version - that is why .bak files are enabled by default. ■ /C{#} Use configuration with ID = # The two predefined configuration names are "default" and "unpack". First is always loaded as lxLite starts and second is used when /X+ option is specified (NOT equivalent to /cUnpack). ■ /D{+|-} Keep (+) or remove (-) [D]ebug info if encountered in executables. Some vendors forget (?) to relink their applications without debug info when it is complete. It will save you some space and will not affect at all the functionality of program itself. By default it is enabled (i.e. removes debug info). ■ /F{+|-} Forced repacking. Use /F+ to bypass `already processed` message, i.e. when lxLite thinks that file was already processed and it really wasn`t. This usually doesn`t happen, but can happen when you try to replace a stub by another of the same size in a already packed file. ■ /M{R{N|1|2|3}|L{N|1}} Set packing method & parameters. Second character defines the method to set-up: `R` stands for run-length (/EXEPACK:1) and 'L' for Lempel-Ziv (/EXEPACK:2). The third character is the level of compression using that method; if N is specified the method is disabled. Three levels of packing are provided for run-length compression. The level 1 is the fastest. It searches only for 1-character strings. For example, the 'AAAAAAAA' string will be detected and packed as 8, 1, 'A' while 'ABABABAB' string will be stored as unpacked text. Level 2 detects strings of any length up to 16 characters long. The example above will be encoded then as 4,2,'AB'. This is the default setting for most 'factory` configurations. And last, 3rd level searchs for ALL strings of any length (up to page size/2 = 2048). This compress VERY slow and seldom gives real results, so use it only when you really need it. The Lempel-Ziv algorithm can be either disabled (/MLN) or enabled (/ML1). When enabled it searchs for all matches using a relatively fast hash-table, so there is no need in gradations by compression speed. ■ /P{+|-} Enable (+) or disable (-) pause before each file. The program shows the name of file which will be processed and offers a choice to continue or to abort. ■ /R{#} [R]e-align pages on specific boundary. (#) must be a power of two. This is an analog of LINK386`s /ALIGN: switch. Many of the programmers don`t care that the default setting for /A: is to align each page inside the executable on 512 (sector) bound. The result is a lot of unused space inside the executable. You can `re-link` such files with another /ALIGN: option. Default is /R:4. To force lxLite to behave like LINK386 you must use /R:512 option. This is equivalent to /ASS :-) option. ■ /S{+|-} Show (+) or don`t show (-) configuration in effect. This is useful for examining which settings are stored into .CFG file, especially for linked configurations (see below). For example lxLite /Cdefault /S will show the default settings. ■ /T{#} Use specified file as new DOS stub. DOS stub is a little DOS .EXE file linked to OS/2`s LX file which is usually used to type an error message in the case if the executable is not run from DOS command line. Usually this looks like: This is an OS/2 executable module Along with lxLite are enclosed two stubs: stub_min.bin and stub_vdm.bin. First is the standard `type-error-and-exit` type, but it is slightly smaller than usual stubs used by various linkers. The second is an stub which starts a new OS/2 session and runs program from it again. If OS/2 is not detected it types the same error message and exits. ■ /U{+|-} Enable (+) or disable (-) unpacking file before packing lxLite know how to unpack any of two methods described, so default option state is enabled. Disable it only if you wish to save time. ■ /V{+|-} Verbose (show alot of file information) This is a switch for curious ones :-) It displays a little information about processed file (to get full information on .LX file use exeHdr.EXE from OS/2 Programmer`s Toolkit). ■ /W{+|-} Enable (+) or disable (-) the [W]riting of resulting file to disk. By default it is enabled, but you can disable it if you wish to examine the executable information (/V+) without having rewritten or repacked file. Also this option can be useful for debugging your options. ■ /X{+|-} e[X]pand (+) or pack (-) given files. Use this switch to decompress files. lxLite can decompress files which has been compressed by itself as well as by other programs which uses standard methods (i.e. Resource Compiler, LINK386, REPACK). It is NOT identical to /cUnpack option. ■ /Z{#} This option will set the `threshold` for lxLite to help him determine when stub is a `dummy` one and when it is a functional program. There are a number of programs (for example, \os2\xdfloppy.exe) which runs both under DOS and OS/2 - in such programs DOS executable is implemented into OS/2`s LX as a DOS stub. By default (and using simply /Z) lxLite considers all stubs bigger than 1024 bytes as functional ones, and therefore on them the /T{#} option has no effect. /?,/H Show a brief help. This is useful when you forget a particular switch from all that list :-) ──────────────────────────────────────────────────────────────────────────── The .CFG file format is as simple as possible: it is a plain ASCII text file which you can edit using any editor which supports them (almost any). Any characters after a semicolumn ';' are ignored, i.e. you can put comments in configuration file like this: ; This is an comment Any line begining with a word and column (ex: "word:") are treated as a particular configuration. This word is used to identify the configuration. This is an example of a configuration entry: failsafe: /APP /B- /G+ /R4 /MR2 /ML1 /P- /U+ /V- As you can see, after column you should write any switches just like you do with lxLite itself. Recursive /C options are NOT ignored, so you can link one configuration to another, but /C option is proceeded last, so from two overlapping options will be in effect the last one. Just for example: here: /app /b- /g+ there: /ass /b+ /p+ Running "lxLite /cThere" will be equivalent to "lxLite /app /b- /p+ /g+" Note that lxLite will ignore any sequenced links to the same configuration to avoid infinite-loop recursion. So, "lxLite /cOne /cOne" will load `One` configuration only ONCE. 5. Error messages ───────────────── Like most normal programs :-) lxLite can eventually generate error messages. Some of them can appear in similar conditions, but caused by different causes. Here is a short list of the most frequent errors: * Invalid configuration file format Self-explaining, I think :-) * Error reading configuration file This error is generated ONLY if the file is not physically readable. The program does not generate an error if the configuration file is missing. * Error writing configuration file Self explaining. * error reading executable This is generated if executable is physically unreadable. * error writing executable This is generated if executable cannot be written onto disk. The cause can be the insufficience of disk space - lxLite does not check for this particular error. * invalid executable file format The file is not in [L]inear [E]xecutable. Note that you can get this message for files with .EXE extension in the cause they are in old, 'New Exe (bwhahaha)' format or DOS executable or winDOS executable. * unsupported executable format revision This error can happen (may be :-) if you try to process an executable with other revision number than 0. OS/2 Warp works only with revision 0, so you will not normally encounter this problem. * invalid word/dword ordering in executable The executable uses little-endian byte order. Seems that it is not for Intel platform machines. * executable target is an unsupported CPU type This happen if the target CPU is other than 286, 386, 486 or P5. * executable target is an unsupported OS This mean that the executable is not for OS/2. The windos 95 uses similar format, but its magic number is not `LX` but rather `LE`, so usualy program will abort with `invalid format` error code. * unknown entry bundle type in executable * unknown page flags in executable * invalid object page detected in executable It`s something about internal structure that lxLite doesnt know about. Please mail me if you encounter such files. * not enough memory to load executable I doubt this error can happen in OS/2 :-) Rather a swap-file full fault will occur. BTW, it`s a bad idea of IBM programmers to trap instead of returning NIL pointer at memory request :-( 6. Todo list ──────────── Here is a list of features I plan to add in some future versions. Any suggestions are welcome, to contact me see the last section. * None - tell me if you wanna something. 7. Known bugs and limitations ───────────────────────────── Here are a list of executables which failed for some reasons to work packed; avoid them. * PMJPEG v1.5. However, even IBM`s REPACK cannot repack it, so I think this is because of either a bug in a linker used to generate it (and its executable has a strange structure) or a kind of debug protection (executable checksum checking?) * Executables generated by VX-REXX reportedly lxLite also does not pack correctly. I will hopefully fix this as soon as I will get a such test executable. 8. Contacting me ──────────────── You can contact me via e-mail at: FIDOnet: 2:5030/84.5 (prefered) e-mail: Andrew Zabolotny@p5.n84.r5030.z2.fidonet Enjoy, _\ndy.